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Abstract — This paper studies the problem of utility maxi- 
mization for clients with delay based QoS requirements in 
wireless networks. We adopt a model used in a previous 
work that characterizes the QoS requirements of clients by 
their delay constraints, channel reliabilities, and delivery 
ratio requirements. In this work, we assume that the utility 
of a client is a function of the delivery ratio it obtains. We 
treat the delivery ratio for a client as a tunable parameter 
by the access point (AP), instead of a given value as in the 
previous work. We then study how the AP should assign 
delivery ratios to clients so that the total utility of all clients 
is maximized. 

We apply the techniques introduced in two previous 
papers to decompose the utility maximization problem 
into two simpler problems, a CLIENT problem and an 
ACCESS-POINT problem. We show that this decompo- 
sition actually describes a bidding game, where clients bid 
for the service time from the AP. We prove that although 
all clients behave selfishly in this game, the resulting equi- 
librium point of the game maximizes the total utility. In 
addition, we also establish an efficient scheduling policy for 
the AP to reach the optimal point of the ACCESS-POINT 
problem. We prove that the policy not only approaches 
the optimal point but also achieves some forms of fairness 
among clients. Finally, simulation results show that our 
proposed policy does achieve higher utility than all other 
compared policies. 

I. Introduction 

We study how to provide QoS to maximize utility for 
wireless clients. We jointly consider the delay constraint 
and channel unreliability of each client. The access point 
(AP) assigns delivery ratios to clients under the delay and 
reliability constraints. This distinguishes our work from 
most other work on providing QoS where the delivery 
ratios to clients are taken as given inputs rather than 
tunable parameters. 

We consider the scenario where there is one AP that 
serves a set of wireless clients. We extend the model 
proposed in a previous work [8] . This model analytically 
describes three important factors for QoS: delay, channel 
unreliability, and delivery ratio. The previous work also 
provides a necessary and sufficient condition for the 
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demands of the set of clients to be feasible. In this work, 
we treat the delivery ratios for clients as variables to 
be determined by the AP. We assume that each client 
receives a certain amount of utility when it is provided 
a delivery ratio. The relation between utility and delivery 
ratio is described by an utility function, which may differ 
from client to client. Based on this model, we study the 
problem of maximizing the total utility of all clients, under 
feasibility constraints. We show that this problem can be 
formulated as a convex optimization problem. 

Instead of solving the problem directly, we apply the 
techniques introduced by Kelly [10] and Kelly, Maulloo, 
and Tan [11] to decompose the problem of system utility 
maximization into two simpler subproblems that describe 
the behaviors of the clients and the AP, respectively. We 
prove that the utility maximization problem can be solved 
by jointly solving the two simpler subproblems. Further, 
we describe a bidding game for the reconciliation between 
the two subproblems. In this game, clients bid for service 
time from the AP, and the AP assigns delivery ratios 
to clients according to their bids, to optimize its own 
subproblem, under feasibility constraints. Based on the 
AP's behavior, each client aims to maximize its own net 
utility, that is, the difference between the utility it obtains 
and the bid it pays. We show that, while all clients behave 
selfishly in the game, the equilibrium point of the game 
solves the two subproblems jointly, and hence maximizes 
the total utility of the system. 

We then address how to design a scheduling policy for 
the AP to solve its subproblem. We propose a very simple 
priority based scheduling algorithm for the AP. This policy 
requires no information of the underlying channel qual- 
ities of the clients and thus needs no overhead to probe 
or estimate the channels. We prove that the long-term 
average performance of this policy converges to a single 
point, which is in fact the solution to the subproblem for 
the AP. Further, we also establish that the policy achieves 
some forms of fairness. 

Our contribution is therefore threefold. First, we for- 
mulate the problem of system utility maximization as a 
convex optimization problem. We then show that this 
problem is amenable to solution by a bidding game. 
Finally, we propose a very simple priority based AP 
scheduling policy to solve the AP's subproblem, that can 
be used in the bidding iteration to reach the optimal point 
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of the system's utility maximization problem. 

Finally, we conduct simulation studies to verify all 
the theoretical results. Simulations show that the per- 
formance of the proposed scheduling policy converges 
quickly to the optimal value of the subproblem for AP. 
Also, by jointly applying the scheduling policy and the 
bidding game, we can achieve higher total utility than all 
other compared policies. 

The rest of the paper is organized as follows: Section 
reviews some existing related work. Section Hill introduces 
the model for QoS proposed in [8] and also summarizes 
some related results. In Section [IVl we formulate the 
problem of utility maximization as a convex program- 
ming problem. We also show that this problem can be 
decomposed into two subproblems. Section describes 
a bidding game that jointly solves the two subproblems. 
One phase of the bidding game consists of each client 
selfishly maximizing its own net profit, and the other 
phase consists of the AP scheduling client transmissions 
to optimize its subproblem. Section El addresses the 
scheduling policy to optimize this latter subproblem. Sec- 
tion I VIII demonstrates some simulation studies. Finally, 
Section IVIIII concludes this paper. 

II. Related Work 

There has been a lot of research on providing QoS over 
wireless channels. Most of the research has focused on 
admission control and scheduling policies. Hou, Borkar, 
and Kumar [8] and Hou and Kumar [9] have proposed 
analytical models to characterize QoS requirements, and 
have also proposed both admission control and scheduling 
policies. Ni, Romdhani, and Turletti [13] provides an 
overview of the IEEE 802.11 mechanisms and discusses 
the limitations and challenges in providing QoS in 802.11. 
Gao, Cai, and Ngan [7], Niyato and Hossain [14], and 
Ahmed [1] have surveyed existing admission control al- 
gorithms in different types of wireless networks. On the 
other hand, Fattah and Leung [6] and Cao and Li [5] 
have provided extensive surveys on scheduling policies 
for providing QoS. 

There is also research on utility maximization for both 
wireline and wireless networks. Kelly [10] and Kelly, 
MauUoo, and Tan [11] have considered the rate control 
algorithm to achieve maximum utility in a wireline net- 
work. Lin and Shroff [12] has studied the same problem 
with multi-path routing. As for wireless networks, Xiao, 
Shroff, and Chong [15] has proposed a power-control 
framework to maximize utility, which is defined as a 
function of the signal-to-interference ratio and cannot 
reflect channel unreliability. Cao and Li [4] has proposed 
a bandwidth allocation policy that also considers channel 
degradation. Bianchi, Campbell, and Liao [2] has studied 
utility-fair services in wireless networks. However, all the 
aforementioned works assume that the utility is only 
determined by the allocated bandwidth. Thus, they do 
not consider applications that require delay bounds. 

III. System Model and Feasibility Condition 

We adopt the model proposed in a previous work [8] to 
capture two key QoS requirements, delay constraints and 



delivery ratio requirements, and incorporating channel 
conditions for users. In this section, we describe the 
proposed model and summarize relevant results of [8]. 

We consider a system with N clients, numbered as 
{1, 2, . . . , A^}, and one access point (AP). Packets for 
clients arrive at the AP and the AP needs to dispatch 
packets to clients to meet their respective requirements. 
We assume that time is slotted, with slots numbered as 
{0,1,2,...}. The AP can make exactly one transmission 
in each time slot. Thus, the length of a time slot would 
include the times needed for transmitting a DATA packet, 
an ACK, and possibly other MAC headers. Assume there is 
one packet arriving at the AP periodically for each client, 
with a fixed period of r time slots, at time slots 0, r, 2t, . . . . 
Each packet that arrives at the beginning of a period 
[fcr, (fc+l)r) must be delivered within the ensuing period, 
or else it expires and is dropped from the system at the 
end of this period. Thus, a delay constraint of r time slots 
is enforced on all successfully delivered packet. Further, 
unreliable and heterogeneous wireless channels to these 
clients are considered. When the AP makes a transmission 
for client n, the transmission succeeds (by which is meant 
the successful deliveries of both the DATA packet and the 
ACK) with probability p„. Due to the unreliable channels 
and delay constraint, it may not be possible to deliver the 
arrived packets of all the clients. Therefore, each client 
stipulates a certain delivery ratio g„ that it has to receive, 
which is defined as the average proportion of periods in 
which its packet is successfully delivered. The previous 
work also shows how this model can be used to capture 
scenarios where both uplink traffic and downlink traffic 
exist. 

Below we describe the formal definitions of the con- 
cepts of fulfilling a set of clients and the feasibility of a 
set of client requirements. 

Definition 1: A set of clients with the above QoS con- 
straints is said to be fulfilled by a particular scheduling 
policy rj of the AP if the time averaged delivery ratio of 
each client is at least with probability 1. 

Definition 2: A set of clients is feasible if there exists 
some scheduling policy of the AP that fulfills it. 

Whether a certain client is fulfilled can be decided by 
the average number of time slots that the AP spends on 
working for the client per period: 

Lemma 1: The delivery ratio of client n converges to 
Qn with probability one if and only if the work performed 
on client n, defined as the long-term average number of 
time slots that the AP spends on working for client n per 
period, converges to Wn{qn) ~ with probability one. 
We therefore call Wn{qn) the workload of client n. 

Since expired packets are dropped from the system at 
the end of each period, there is exactly one packet for 
each client at the beginning of each period. Therefore, 
there may be occasions where the AP has delivered all 
packets before the end of a period and is therefore forced 
to stay idle for the remaining time slots in the period. Let 
Is be the expected number of such forced idle time slots 
in a period when the client set is just S C {1,2,..., N} 
(i.e., all clients except those in S are removed from 
consideration), and the AP only caters to the subset S 
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of clients. Since each client n £ S requires u;„ time 
slots per period on average, we can obtain a necessary 
condition for feasibility: X^ies + ^ t, for all 

S C {1,2,..., N}. It is shown in [8] that this necessary 
condition is also sufficient: 

Theorem 1 : A set of clients, with delivery ratio require- 
ments [qn], is feasible if and only if J2ies ~ ^ Is, for 
all 5 C {1,2,..., TV}. 

IV. Utility Maximization and Decomposition 

In the previous section, it is assumed that the delivery 
ratio requirements, [qn], are given and fixed. In this paper, 
we address the problem of how to choose q := [qn] so 
that the total utility of all the clients in the system can be 
maximized. 

We begin by supposing that each client has a certain 
utility function, Un{qn), which is strictly increasing, 
strictly concave, and continuously differentiable function 
over the range < < 1, with the value at defined 
as the right limit, possibly — oo. The problem of choosing 
qn to maximize the total utility, under the feasibility 
constraint of Theorem [l] can be described by the 
following convex optimization problem: 



SYSTEM: 



Max J:^=lU^{q^) 

s-t. <^-^s,V5C{l,2,...,iV}, 

Pi 

over qn >0,yi<n<N. 



(1) 
(2) 

(3) 



It may be difficult to solve SYSTEM directly So, 
we decompose it into two simpler problems, namely, 
CLIENT and ACCESS-POINT, as described below. 
This decomposition was first introduced by Kelly [10], 
though in the context of dealing with rate control for non- 
real time traffic. 

Suppose client n is willing to pay an amount of p„ per 
period, and receives a long-term average delivery ratio 
qn proportional to p„, with p„ = ^„g„. If > 0, the 
utility maximization problem for client n is: 



CLIENT„ 



Max C/„(^)-p„ 

over 0< Pn < Ipn- 



(4) 
(5) 



On the other hand, given that client n is willing to 
pay pn per period, we suppose that the AP wishes to 
find the vector q to maximize J2f=i Pi^'^SQi, under the 
feasibility constraints. In other words, the AP has to solve 
the following optimization problem: 



ACCESS-POINT: 



Max E»=i/0» log* 
over qn > 0,V1 <n<N. 



<T-/s,V5C{l,2,...,iV}, 



(6) 
(7) 
(8) 



We begin by showing that solving ACCESS-POINT 
is equivalent to jointly solving CLIENTn and ACCESS- 
POINT. 

Theorem 2: There exist non-negative vectors q, p := 
[pn], and V' := with p„ = '4'nq7i, such that: 

(i) For n such that ipn > 0, pn is a solution to 

CLIENTn, 

(ii) Given that each client n pays p„ per period, g is a 
solution to ACCESS-POINT. 

Further, if q, p, and ip are all positive vectors, the vector 
q is also a solution to SYSTEM. 

Proof: We will first show the existence of q, p, and 
ip that satisfy ^ and dn]). We will then show that the 
resulting q is also the solution to SYSTEM. 

There exists some e > so that by letting qn = e, 
the vector q is an interior point of the feasible region 
for both SYSTEM © ©, and ACCESS-POINT ^ 
([8]). Also, by setting p„ = e, p„ is also an interior 
point of the feasible region for CLIENTn Therefore, 
by Slater's condition, a feasible point for SYSTEM, 
CLIENTn, or ACCESS-POINT, is the optimal solution 
for the respective problem if and only if it satisfies the 
corresponding Karush-Kuhn-Tucker (KKT) condition for 
the problem. Further, since the feasible region for each of 
the problems is compact, and the utilities are continuous 
on it, or since the utility converges to — cx) at qn = 0, there 
exists an optimal solution to each of them. 

The Lagrangian of SYSTEM is: 



LsYs{q, A, v) := - Yln^i Ui{qi) 

+ Esc{i,2,...,w} AsEiss f,-'y'^~ ^s)] 



where A := [A5 : 5" C {1, 2, . . . , N}] and := K : 1 < n < 
N] are the Lagrange multipliers. By the KKT condition, 
a vector q* :— [q^, • ■ • 7 ^at] '^he optimal solution to 
SYSTEM if q* is feasible and there exists vectors A* and 
V* such that: 



dL 



SYS 



= -[/' (q*) + ^^3"^^ 



q* ,A* ,1/ 

^1 



(9) 



ies 



Pi 



= 0,V1 < n < /V, 
(r-/s)] =0,V5C {1,2,..., TV}, (10) 



<g,: = 0,Vl<7i<TV, (11) 
A5 > 0,V5 C {1, . . .,7V}, and v*^ > 0,V1 <n<N. (12) 

The Lagrangian of CLIENTn is: 

LcLl{Pn,in) ~Uni^) + Pn - inPn, 

where ^„ is the Lagrange multiplier for CLIENTn. By the 
KKT condition, p* is the optimal solution to CLIENTn if 
Pn > and there exists ^* such that: 



dL 



CLI 



dp. 

CPn = 0; 



= — —U'( — ) + 1 - f* =0 



(13) 

(14) 
(15) 
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Finally, the Lagrangian of ACCESS-POINT is: 



LNETiq,C,f^) 



+ EsC{l,2,...,Ar} CsEj6S " ('^ ~ -^s)] " E^^l 

where C := [Cs : ^ C {1, 2, . . . , TV}] and ^ := [/i„ : 
1 < n < A^] are the Lagrange multipliers. Again, by the 
KKT condition, a vector q* := [g*] is the optimal solution 
to ACCESS-POINT if q* is feasible and there exists 
vectors (* and ^* such that: 



_ p.. 

9,* 



(16) 



= 0,V1 < n < iV, 
(T-/5)] =0,V5C {1,2,. 



(17) 



(18) 



« = 0,Vl<n<iV, 
Cs > 0,V5 C {1, . . .,N}, and > 0,V1 < n < iV. (19) 

Let q* be a solution to SYSTEM, and let A*, j/* 
be the corresponding Lagrange multipliers that satisfy 

conditions ©-([El). Let q„ 



and 



A* 



, for all 71. Clearly, q, p, and tp are all non- 



negative vectors. We will show {q, p, ip) satisfy ^ and dn]). 
We first show © for all n such that V'n 



^ > 0. 

It is obvious that p„ = V'n'i'n- Also, p„ > 0, since > 
(by (1121) ) and g* > (since q* is feasible). Further, let 
the Lagrange multiplier of CLIENTn, be equal to 



K/ 



OLI 



i^'^/i/jn- Then we have: 



= —l~U' 



■0n - 



CnPn = -r-lpnqn 



0, by dn]) 



<) = 0, by dlD, 



S3n -^'S 



> 0, by (fT2l) . 



In sum, (/o, i/'jC) satisfies the KKT conditions for 
CLIENTn, and therefore /9„ is a solution to CLIENTn, 
with p„ = V'ngn- 

Next we establish dn]). Since g = is the solution 
to SYSTEM, it is feasible. Let the Lagrange multipliers 
of ACCESS-POINT be Cs = A^,V5, and ^„ = 0,Vn, 
respectively. Given that each client n pays p„ per period, 
we have: 



a9„ 



9,1 
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= -V^n + V'n - = 0,Vn, 

= 0,VS', by'dlO]), 

IJ^nqn = X qn = 0, Vn, 

Cs = As > 0,VS' (by dEl)), and /i„ > 0,Vn 



Therefore, {q, (, fi) satisfies the KKT condition for 
ACCESS-POINT and thus q is a solution to ACCESS- 
POINT. 

For the converse, suppose {q, p, ^) are positive vectors 
with pn = V'n'Zn, for all n, that satisfy dll) and dUl)- We 
wish to show that g is a solution to SYSTEM . Let ^„ be 
the Lagrange multiplier for CLIENTn- Since we assume 
tpn > for all n, the problem CLIENTn is well-defined 
for all n, and so is Also, let ( and ^ be the Lagrange 
multipliers for ACCESS-POINT. Since g„ > for all n, 
we have /z„ = for all n by dlSP . By (I16t . we also have: 



p.. . 



Pre 

Ss3re Cs 
Pre 



= 0, 



and thus -0„ = — . Let As = Cs, for all 5*, and Vn = 
TpnS.n, for all n. We claim that q is the optimal solution to 
SYSTEM with Lagrange multipliers A and v. 

Since g is a solution to ACCESS-POINT, it is feasible. 
Further, we have: 



dLs 



-U'ni^) + - V-nC™ = 0,Vn, by dH, 



AsE„,slr-(^-^s)] =CsE„eslr"(^-^^ 



0,V5, by dni), 



l^nqn = £.nPn = 0,Vn, by dS]) , 

As-Cs>0,V5, by dUl), 
= i^n^n > 0, Vn, by dUl). 

Thus, ((?, A, i^) satisfy the KKT condition for SYSTEM, 
and so q is a solution to SYSTEM. ■ 

V. A Bidding Game between Clients and Access 
Point 

Theorem |2] states that the maximum total utility of 
the system can be achieved when the solutions to the 
problems CLIENTn and ACCESS-POINT agree. In 
this section, we formulate a repeated game for such 
reconciliation. We also discuss the meanings of the prob- 
lems CLIENTn and ACCESS-POINT in this repeated 
game. 

The repeated game is formulated as follows: 

Step 1: Each client n announces an amount p„ that it 
pays per period. 

Step 2: After noting the amounts, pi,p2, ■ ■ ■ , pn, paid by 
the clients, the AP chooses a scheduling policy so 
that the resulting long-term delivery ratio, for 
each client maximizes J2iLi Pi log<Zi. 

Step 3: The client n observes its own delivery ratio, g„. It 
computes tpn ■= Pn/qn- It then determines Pn>Q 
to maximize Un{^) — Pn- Client n updates the 
amount it pays to {l — a)pn+apn, with some fixed 
< a < 1, and announces the new bid value. 

Step 4: Go back to Step 2. 

In Step 3 of the game, client n chooses its new amount 

of payment as a weighted average of the past amount and 
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the derived optimal value, instead of the derived optimal 
value. This design serves two purposes. First, it seeks to 
avoid the system from oscillating between two extreme 
values. Second, since p„ is initiated to a positive value, 
and p* derived in each iteration is always non-negative, 
this design guarantees p„ to be positive throughout all 
iterations. Since -0„ — pn/qn, this also ensures ijjn > and 
the function C/„(^) is consequently always well-defined. 

We show that the fixed point of this repeated game 
maximizes the total utility of the system: 

Theorem 3: Suppose at the fixed point of the repeated 
game, each client n pays p* per period, and receives 
delivery ratio g*. If both p* and g* are positive for all 
n, the vector q* maximizes the total utility of the system. 

Proof: Let = ^ . It is positive since both p* and g* 
are positive. Since the vectors q* and p* are derived from 
the fixed point, p* maximizes Un{^)—pn, over all p„ > 0, 
as described in Step 3 of the game. Thus, p* is a solution 
to CLIENTn, given p* = i^nln- Similarly, from Step 2, q* 
is the feasible vector that maximizes J2iLi Pi ^ogqi, over 
all feasible vectors q. Thus, q* is a solution to ACCESS- 
POINT, given that each client n pays p* per period. By 
Theorem [21 q* is the unique solution to SYSTEM and 
therefore maximizes the total utility of the system. ■ 

Next, we describe the meaning of the game. In Step 
3, client n assumes a linear relation between the amount 
it pays, p„, and the delivery ratio it receives, g„. To be 
more exactly, it assumes p„ = i^nQn, where ipn is the price. 
Thus, maximizing C/„(^)— p„ is equivalent to maximizing 
Un{qn) — Pn- Recall that Un{qn) is the utility that client 
n obtains when it receives delivery ratio [/„((/„) — p„ 
is therefore the net profit that client n gets. In short, in 
Step 3, the goal of client n is to selfishly maximize its own 
net profit using a first order linear approximation to the 
relation between payment and delivery ratio. 

We next discuss the behavior of the AP in Step 2. The 
AP schedules clients so that the resulting delivery ratio 
vector g is a solution to the problem ACCESS-POINT, 
given that each client n pays p„ per period. Thus, q 
is feasible and there exists vectors Q and p that satisfy 
conditions (fT6])-([T9l). While it is difficult to solve this 
problem, we consider a special restrictive case that gives 
us a simple solution and insights into the AP's behavior. 
Let TOT := {1, 2, . . . , TV} be the set that consists of all 
clients. We assume that a solution (g, C, p) to the problem 
has the following properties: Qs — 0, for all S ^ TOT, 
(tot > 0, and pn — 0, for all ri. By (1161) . we have: 

_ Pn Y^SBn (S _ _ _ p„ CtOT _ ^ 
I Pn — I — 

Qn Pn Qn Pn 

and therefore (?„ = PnPn/CroT- Further, since Ctot > 0, 
( |17D requires that: 

^ — - (r - Itot) = ^ -T^ (t - Itot) = 0. 

ieTOT ieTOT ^'^OT 

Thus, Ctot = and ^ = ^^(^ " ^tot), for 

all n. Notice that the derived (g, (, p) satisfies conditions 
(ll6l) -([T9l). Thus, under the assumption that g is feasible. 



this special case actually maximizes Pi log9i- 

Section |^ we will address the general situation without 
any such assumption, since it needs not be true. 

Recall that Itot is the average number of time slots 
that the AP is forced to be idle in a period after it 
has completed all clients. Also, by Lemma [T] ^ is the 
workload of client n, that is, the average number of time 
slots that the AP should spend working for client n. Thus, 
letting ^ = ^jC" p. [r - Itot), for all n, the AP tries 
to allocate those non-idle time slots so that the average 
number of time slots each client gets is proportional to its 
pajnnent. Although we only study the special case of Itot 
here, we will show that the same behavior also holds for 
the general case in the Section fVll 

In summary, the game proposed in this section actually 
describes a bidding game, where clients are bidding for 
non-idle time slots. Each client gets a share of time slots 
that is proportional to its bid. The AP thus assigns delivery 
ratios, based on which the clients calculate a price and 
selfishly maximize their own net profits. Finally, Theorem 
[3] states that the equilibrium point of this game maximizes 
the total utility of the system. 

VI. A Scheduling Policy for Solving 
ACCESS-POINT 

In Section we have shown that by setting g„ = 
Pn j^'^'^m ~ Itot), the resulting vector g solves 
ACCESS-POINT provided g is indeed feasible. Un- 
fortunately, such g is not always feasible and solving 
ACCESS-POINT is in general difficult. Even for the 
special case discussed in Section |Vl solving ACCESS- 
POINT requires knowledge of channel conditions, that 
is, Pn. In this section, we propose a very simple priority 
based scheduling policy that can achieve the optimal 
solution for ACCESS-POINT, and that too without any 
knowledge of the channel conditions. 

In the special case discussed in Section |Vl the AP tries, 
though it may be impossible in general, to allocate non- 
idle time slots to clients in proportion to their payments. 
Based on this intuitive guideline, we design the following 
scheduling policy. Let u„(t) be the number of time slots 
that the AP has allocated for client n up to time t. At 
the beginning of each period, the AP sorts all clients in 
increasing order of "^hiJA so that < < ... after 

° Pn ' Pi — P2 — 

renumbering clients if necessary. The AP then schedules 
transmissions according to the priority ordering, where 
clients with smaller ^^^j^ get higher priorities. Specifically, 
in each time slot during the period, the AP chooses the 
smallest i for which the packet for client i is not yet 
delivered, and then transmits the packet for client i in 
that time slot. We call this the weighted transmission policy 
(WT) . Notice that the policy only requires the AP to keep 
track of the bids of clients and the number of time slots 
each client has been allocated in the past, followed by a 
sorting of "^^^^ among all clients. Thus, the policy requires 
no information on the actual channel conditions, and is 
tractable. Simple as it is, we show that the policy actually 
achieves the optimal solution for ACCESS-POINT. In 
the following sections, we first prove that the vector of 
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delivery ratios resulting from the WT policy converges to 
a single point. We then prove that this limit is the opti- 
mal solution for ACCESS-POINT. Finally we establish 
that the WT policy additionally achieves some forms of 
fairness. 

A. Convergence of the Weighted Transmission Policy 

We now prove that, by applying the WT policy, the de- 
livery ratios of clients will converge to a vector q. To do so, 
we actually prove the convergence property and precise 
limit of a more general class of scheduling policies, which 
not only consists of the WT policy but also a scheduling 
policy proposed in [8] . The proof is similar to that used in 
[8] and is based on Blackwell's approachability theorem 
[3] . The proof in [8] only shows that the vector of delivery 
ratios approaches a desirable set in the A^-space under a 
particular policy, while here we prove that the vector of 
delivery ratios converges to a single point under a more 
general class of scheduling policies. Thus, our result is 
both stronger and more general than the one in [8]. 

We start by introducing Blackwell's approachability the- 
orem. Consider a single player repeated game. In each 
round i of the game, the player chooses some action, 
a{i), and receives a reward v{i), which is a random 
vector whose distribution is a function of a{i). Blackwell 
studies the long-term average of the rewards received, 
linij^oc defining a set as approachable, under 

some policy, if the distance between X]i=i ^^id the 

set converges to with probability one, as j oo. 

Theorem 4 (Blackwell [3]): Let ^ C be any closed 
set. Suppose that for every x ^ A, a policy 77 chooses 
an action a {— a{x)), which results in an expected payoff 
vector E{v). If the h5^erplane through y, the closest point 
in A to X, perpendicular to the line segment xy, separates 
X from E{v), then A is approachable with the policy r]. 

Now we formulate our more general class of scheduling 
policies. We call a policy a generalized transmission time 
policy if, for a choice of a positive parameter vector a and 
non-negative parameter vector b, the AP sorts clients by 
o-nUn{t) — bnt at the beginning of each period, and gives 
priorities to clients with lower values of this quantity. Note 
that the special case a„ = and &„ = yields the WT 
policy, while the choice a„ = 1 and 6„ = yields the 
largest time-based debt first policy of [8], and thus we 
describe a more general set of policies. 

Theorem 5: For each generalized transmission time pol- 
icy, there exists a vector q such that the vector of work 
loads resulting from the policy converges to w{q) := 

[■Wniqn)]- 

Proof: Given the parameters {(a„,6„) : I < n < N}, 
we give an exact expression for the limiting q. We define 
a sequence of sets {Hk} and corresponding values {6k} 
iteratively as follows. Let Hq :— (f), 60 :— —00, and 



Hk '■ = arg min 



- Is) - E, 



7fc :=- 



for all fc > 0. 



In selecting Hk, we always choose a maximal subset, 
breaking ties arbitrary. 6'i), (iJ2, ^'2), ■ • • , can be it- 
eratively defined until every client is in some Hk- Also, 
by the definition, we have 6k > 6k-i, for all fc > 0. If 
client n is in Hk\Hk-i, we define qn rpn ^"^^'^ , and 
so Wn{qn) = r ''" +^'' . The proof of convergence consists 
of two parts. First we prove that the vector of work 
performed (see Lemma [T] for definition) approaches the 
set {w*\w^ > Wniqn)}- Then we prove that w(q) is the 
only feasible vector in the set {w*|w* > w„((7„)}. Since 
the feasible region for work loads, defined as the set of 
all feasible vectors for work loads, is approachable under 
any policy, the vector of work performed resulting from 
the generalized transmission time policy must converge 
to w{q). 

For the first part, we prove the following statement: for 



each fc > 1, the set Wk := {w*\w*„ > T^^^^^^n ^ Hk^i} 
is approachable. Since HiyoWi = {w*\w*^ > it;„(q„)}, we 
also prove that {w*\'w*^ > t«„(g„)} is approachable. 

Consider a linear transformation on the space of work- 
loads L{w) := [In : In = °""'^"''" ]. Proving Wk is 
approachable is equivalent to proving that its image under 
L, Vk {l\ln > ^'Vn ^ Hk-i}, is approachable. 
Now we apply Blackwell's theorem. Suppose at some 
time t that is the beginning of a period, the number 
of time slots that the AP has worked on client n is 



t/r 



and 



Un{t). The work performed for client n is 
the image of the vector of work performed under L 



is x{t) := [Xn{t)\Xn{t) 



which we shall 



suppose is not in Vk- The generalized transmission time 
policy sorts clients so that aiui(<)— 61 < a2U2{t)—b2 < 
or equivalently ,Jaixi{t) < ^/a2X2it) < .... The closest 
point in Vk to x{t) is y [y„], where y„ = if 

Xnit) < and n ^ Hk-i, and y„ = a;„, otherwise. 
The hyperplane that passes through y and is orthogonal 
to the line segment xy is: 



n:n<no .n4Hk 



{Zn 



= 0}. 



Let 7r„ be the expected number of time slots that the AP 
spends on working for client n in this period under the 
generalized transmission time policy. The image under L 
of the expected reward in this period is ttl := [ °"^^^~ — ]. 
Blackwell's theorem shows that Vk is approachable if x{t) 
and ttl are separated by the plane {z\f{z) = 0}. Since 
f{x{t)) > 0, it suffices to show /(ttl) < 0. 

We manipulate the original ordering, for this period, 
so that all clients in Hk-i have higher priorities than 
those not in Hk-i, while preserving the relative ordering 
between clients not in Hk-i- Note this manipulation will 
not give any client n ^ Hk-i higher priority than it had in 
the original ordering. Therefore, 7r„ will not increase for 
any n ^ Hk-i- Since the value of /(ttl) only depends on 
7r„ for n <^ Hk-i, and increases as those 7r„ decrease, this 
manipulation will not decrease the value of /(ttl). Thus, it 
suffices to prove that /(ttl) < 0, under this new ordering. 
Let no := \Hk~.i \ + 1. Under this new ordering, we have: 
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h < ^a„i+ia;„i+i(t) < 



Let(5„ = \/%ra;„(i)-^a„+ia;„+i(<), forno < n < ni-1 
and i5„, 

— \/o^Xni{t) — 9k- Clearly, 6^ 0, for all uq < 



n < ni- Now we can derive: 



ni 

n— no 
ni 

E( 



)(a;n(t) 



-)(\/a^a;„(i) - 6**;) 



L^n—r. 



E 



Recall that Is is the expected number of idle time slots 
when the AP only caters on the subset S. Thus, under this 



ordering, we have X]ri=i 

X],i=no ~ -^{l,...,no-l} 



= T-h 



, for all i, and 



for 

all i > no- By the definition of Hk and 6k, we also have 



Z_-/n— n 



E 

n—7iQ 



bn_ 

an 



1 



c) >0. 



Therefore, /(tt^) < 0, since (5^ < 0, and Vk is indeed 
approachable, for all fc. 

We have established that the set {w*\w^ > w„(g„)} 
is approachable. Next we prove that [w„ ((/„)] is the only 
feasible vector in the set. Consider any vector w' ^ w{q) 
in the set. We have w^^ > w„(g„) for all n, and w[^^ > 
w„Q ((/„(,), for some hq- Suppose no e Hk\Hk-i- We have: 



E 



W' > 



neH, 



w„{qn 



k 

E E 

i=l neHi\Hi 



bn+dk 



and thus w' is not feasible. Therefore, w{q) is the only 
feasible vector in {w*\w* > Wn{qn)}, and the vector of 
work performed resulting from the generalized transmis- 
sion time policy must converge to ^(q). ■ 
Corollary 1 : For the policy of Theorem [S] the vector of 
delivery ratios converges to q. 

Proof: Follows from Lemma [TJ ■ 

B- Optimality of the Weighted Transmission Policy for 
ACCESS-POINT 

Theorem 6: Given [p„], the vector q of long-term av- 
erage delivery ratios resulting from the WT policy is a 
solution to ACCESS-POINT- 

Proof: We use the sequence of sets {Hk} and values 
{9k}, with a„ := — and &„ := 0, as defined in the proof of 
Theorem[5j Let //"= \{9k}\- Thus, we have Hk = TOT = 
{1,2,..., N}- Also, let rrik :~ \Hk\- For convenience, we 
renumber clients so that Hk ~ {1,2,... ,mk}- The proof 
of TheoremOshows that qn = Tpn9kPn, for n G Hk\Hk-i- 
Therefore, w„((?„) = ^ — T9kPn- Obviously, q is feasible. 



since it is indeed achieved by the WT policy. Thus, to 
establish optimality, we only need to prove the existence 
of vectors Q and ^ that satisfy conditions (I16D -([T9]). 
Set ^„ — 0, for all n. Let — Ctot — — 

and Ch. "-^ 



t«jv(<;iv) t9k 
1 1 

T0k 



for 1 < k < K - 1. Finally let Cs := 0, for all 
S ^ {Hi, H2, ■ ■ . , Hk}- We claim that the vectors C and 
p, along with q, satisfy conditions (|16D -([T9]). 

We first evaluate condition (I16D . Suppose client n is in 
Hk\Hk-i- Then client n is also in Hk+i,Hk+2, ■ • • , Hk- 
So, 



qn 



Y.S3n Cs 



Pn 



0. 



T9kPn 



J2f=k (Hi 
Pn 



T9kPn T9kPn 

Thus, condition (1161) is satisfied. 

Since — 0, for all n, condition ( I18D is satisfied. Fur- 
ther, since J- > for all 1 < fc < iiT-l, condition ( [T9] ) 



is also satisfied. It remains to establish condition gi7D . 
Since Cs = for all 5 ^ {Hi,H2, . . . , iJ/^}, we only need 
to show^,g5|-(T-/5) ^Ofor^e {i?i,i/2,...,ifK}. 

Consider Hk - For each client i e iJfc and each client j ^ 
^^fc, ^"p^''' < ^^^^i^. Since Wn{qn) is the average number 
of time slots that the AP spends on working for client n, 
we have ^ < for all i e Hk and j i Hk, after 

a finite number of periods. Therefore, except for a finite 
number of periods, clients in Hk will have priorities over 
those not in Hk- In other words, if we only consider the 
behavior of those clients in Hk, it is the same as if the AP 
only works on the subset Hk of clients. Further, recall that 
///^ is the expected number of time slots that the AP is 
forced to stay idle when the AP only works on the subset 
Hk of clients. Thus, we have J2ieH = t — Ir^ and 

T.^eH,f^-i^-lH,) = Q,ior^\\k. ■ 

C. Fairness of Allocated Delivery Ratios 

We now show that the WT policy not only solves the 
ACCESS-POINT problem but also achieves some forms 
of fairness among clients. Two common fairness criteria 
are max-min fair and proportionally fair- We extend the 
definitions of these two criteria as follows: 

Definition 3: A scheduling policy is called weighted 
max-min fair with positive weight vector a — [an] if it 
achieves q, and, for any other feasible vector q', we have: 
q( > qi^ q'^ < q^, for some j such that ^^^^^ > H^M. 

Definition 4: A scheduling policy is called weighted pro- 
portionally fair with positive weight vector a if it achieves 
q and, for any other feasible vector q', we have: 

y.N Wnjq'n) - Wnjqn) ^ ^ 
""^ Wn{qn)/an ~ 

Next, we prove that the WT policy is both weighted 
max-min fair and proportionally fair with weight vector 
P- 

Theorem 7: The weighted transmission policy is 
weighted max-min fair with weight p 
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Proof: We sort clients and define {Hk} as in the proof 
of Theorem (6) Let q be the vector achieved by the WT 
poHcy and q' be any feasible vector Suppose > qi 
for some i. Assume client i is in Hk\Hk-i- The proof 
in Theorem [6] shows that "^"('Jn) = t - Ir^ - 

On the other hand, the feasibility condition requires 
T.neH, wn{q'n) < T - Ih, T^nEH, Wn{qn). Further, since 
q[ > qi, Wi{q'j) > Wi{qi), there must exist some j e Hk 
so that Wj{q'j) < Wj{qj), that is, q'j < qj. Finally, since 

I e Hk\Hk-i, we have ^ > for all n e Hk, 

and hence > " " ■ 

Pi — P] 

Theorem 8: The weighted transmission policy is pro- 
portionally fair with weight p. 

Proof: We sort clients and define {Hk} as in the proof 
of Theorem \6\ Let q be the vector achieved by the WT 
policy, and let q' be any feasible vector We have Hi(2il = 
Tdk, if I e Hk\Hk-i. Define Afc EneH,\H,^^ ^«(9n) - 

Wniq-n)- 

To prove the theorem, we prove a stronger statement 
by induction: 



E 



Wnjqn) - Wnjqn) 
W„{qn)/Pn 



A 



V — f < 0, for all fc > 0. 



First consider the case k = 1. The proof in Theorem 
[6] shows that Y.neHi'^n(,qn) = t - Iri- Further, the 
feasibility condition requires X]ne//i '^'"(^n) — ~ ^ 
EneHi u'nC^n) ^T- In,, and so Ai J2neH^ (<?■«) - 
Wrilgn) < 0. Thus, we have < 0. 

Suppose we have J2i=i — 0' ^'^^ k < ko- Again, 
the proof in Theorem [6] gives us J^neH^ +i ^"(^n) = 
r — Ih^^+i and the feasibility condition requires 

E„eff.o + i WnWn) < T-Ih,„^i = + i Wn{qn). ThuS, 

Ef=t^ A, < 0. We can further derive: 

fco+l 



V — 

^ re, 

^0 A- 

z— 1 



'j+i 



7/co + l 



feo + 1 

) (since ^ 



A,: 



fco + l 



<0) 



7fco + l 



A^ 



A 



<0 (since E ~^ - ^-J+i > ^^ '^-^ - ^o) 



By induction, < foi' all k. Finally, we have: 



N 

E 



Wn{q'n) ~ Wnjqn 
Wn{qn)/Pn 



^ A, 



■i— 1 

and the WT policy is proportionally fair with weight p. ■ 

VII. Simulation Results 

We have implemented the WT policy and the bidding 
game, as described in Section |Vl on ns-2. We use the 




Fig. 1 : Convergence of the weighted transmission policy 



G.711 codec for audio compression to set the simulation 
parameters, as summarized in Tabled All results in this 
section are averages of 20 simulation runs. 

TABLE I: Simulation Setup 



Packetization interval 


20 ms 


Payload size per packet 


160 Bytes 


Transmission data rate 


11 Mb/s 


Transmission time (including MAC overheads) 


610 us 


# of time slots in a period 


32 



A. Convergence Time for the Weighted Transmission Policy 

We have proved that the vector of delivery ratios will 
converge under the WT policy in Section IVI-AI However, 
the speed of convergence is not discussed. In the bidding 
game, we assume that the delivery ratio observed by each 
client is post convergence. Thus, it is important to verify 
whether the WT policy converges quickly. In this simula- 
tion, we assume that there are 30 clients in the system. 
The n*'* client has channel reliability (50 + n)% and offers 
a bid pn = {n mod 2) + 1. We run each simulation for 10 
seconds simulation time and then compare the absolute 
difference of ^„ Pn^ogqn between the delivery ratios at 
the end of each period with those after 10 seconds. In 
particular, we artificially set g„ = 0.001 if the delivery 
ratio for client n is zero, to avoid computation error for 
log(?„. 

Simulation results are shown in Fig. [TJ It can be seen 
that the delivery ratios converge rather quickly. At time 
0.2 seconds, the difference is smaller than 1.4, which is 
less than 10% of the final value. Based on this observation, 
we assume that each client updates its bid every 0.2 
seconds in the following simulations. 

B. Utility Maximization 

In this section, we study the total utility that is achieved 
by iterating between the bidding game and the WT policy, 
which we call WT-Bid. We assume that the utility function 
of each client n is given by Tn '^"^""^ , where 7„ is a 
positive integer and < a„ < 1. This utility function is 
strictly increasing, strictly concave, and differentiable for 
any 7,, and a„. In addition to evaluating the policy WT- 
Bid, we also compare the results of three other policies: a 



9 



-11 
-13 
-15 
-17 
-19 
-21 



-WT-Bid 
- WT-NoBid 
-Rand 
-P-Rand 



2 3 
time (s) 



(a) Average of total utility 




-WT-Bid 
■WT-NoBid 
-Rand 
-P-Rand 
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Fig. 2: Performance of total utility 



policy that employs the WT policy but without updating 
the bids from clients, which we call WT-NoBid; a policy 
that decides priorities randomly among clients at the 
beginning of each period, which we call Rand; and a 
policy that gives clients with larger 7„ higher priorities, 
with ties broken randomly, which we call P-Rand. 

In each simulation, we assume there are 30 clients. The 
rt*'* client has channel reliability p„ = (50 + n)%, 7„ = [n 
mod 3) + 1, and q;„ — 0.3 + 0.1(7i mod 5). In addition 
to plotting the average of total utility over all simulation 
runs, we also plot the variance of total utility. 

Fig. [2] shows the simulation results. The WT-Bid policy 
not only achieves the highest average total utility but also 
the smallest variance. This result suggests that the WT- 
Bid policy converges very fast. On the other hand, the 
WT-NoBid policy fails to provide satisfactory performance 
since it does not consider the different utility functions 
that clients may have. The P-Rand policy offers much 
better performance than both the WT-NoBid policy and 
the Rand policy since it correctly gives higher priority 
to clients with higher 7„. Still, it cannot differentiate 
between clients with the same 7„ and thus can only 
provide suboptimal performance. 

VIII. Concluding Remarks 

We have studied the problem of utility maximization 
problem for clients that demand delay-based QoS support 
from an access point. Based on an analj^ical model for 
QoS support proposed in previous work, we formulate 



the utility maximization problem as a convex optimiza- 
tion problem. We decompose the problem into two sim- 
pler subproblems, namely CLIENT^ and ACCESS- 
POINT. We have proved that the total utility of the 
system can be maximized by jointly solving the two sub- 
problems. We also describe a bidding game to reconciliate 
the two subproblems. In the game, each client announces 
its bid to maximize its own net profit and the AP allocates 
time slots to achieve the optimal point of ACCESS- 
POINT. We have proved that the equilibrium point of 
the bidding game jointly solves the two subproblems, and 
therefore achieves the maximum total utility. 

In addition, we have proposed a very simple, priority- 
based weighted transmission policy for solving the 
ACCESS-POINT subproblem. This policy does not re- 
quire that the AP know the channel reliabilities of the 
clients, or their individual utilities. We have proved that 
the long-term performance of a general class of priority- 
based policies that includes our proposed policy converges 
to a single point. We then proved that the limiting point of 
the proposed scheduling policy is the optimal solution to 
ACCESS-POINT. Moreover, we have also proved that 
the resulting allocation by the AP satisfies some forms of 
fairness criteria. Finally, we have implemented both the 
bidding game and the scheduling policy in ns-2. Simu- 
lation results suggests that the scheduling policy quickly 
results in convergence. Further, by iterating between the 
bidding game and the WT policy, the resulting total utility 
is higher than other tested policies. 
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